Computers and computerized devices have become increasingly common and increasingly powerful in today's world, enabling automation of tasks previously performed by humans, and in addition performing or facilitating some tasks that were previously impossible. However, as powerful as computers are, the process of creating the software typically used by computers is still a very human-oriented process. Typically, a designer first creates a specification that describes the desired functionality of a piece of software. Subsequently, a developer writes computer code, typically in source code form, to instruct the computer as what operations to perform, actions to take, etc. Once the source code is complete, computer-readable code may be created therefrom. As will be appreciated by those of skill in the art, this process of converting source code into a computer-readable form is often referred to as “compiling.”
Once the compiled version of the code has been produced, the first stage of development is over, but the software is typically not yet ready to be released to the public or other intended consumer. Rather, a testing process is now initiated to identify and correct any bugs, errors, or flaws in the software. At this stage in development, the types of problems that are addressed are typically coding or implementation errors, rather than errors in usability, or other similar errors that pertain more to the overall design of the software as embodied in the specification.
The testing process is an expensive and time-consuming process. Typically an individual called a “tester” tests specific use scenarios with respect to the software. This may be done in a number of ways. One way is for the tester to actually operate the software in the same manner that an end-user would, trying out many different types of operations, keystrokes, menu selections, etc. During this process, the tester observes the behavior of the software, as executed by the computer, to identify any odd or incorrect behavior. The tester reports any problems to the developers, who in turn typically will rewrite a portion of the source code to correct the problem. It will be appreciated that this method of testing is expensive and time consuming. Moreover, the test space, or the group of all actions and combinations thereof that the tester could possibly test, is enormous. Given the complexity of present software products, the test space is essentially unbounded, i.e. due to time or other constraints, a human tester will typically not be able to test every action or combination of actions that is in the test space, and moreover may not even be aware of many or most such actions or combination of actions.
Another method often used to test software involves the production of test code. In particular, an individual familiar with the target software and its intended operation can write test code that will manipulate the software in the same way that might result from actual use of the software by an end user. Typically, the test code functions at a fairly low level, interfacing directly with APIs rather than interfacing with the system from the high level at which an actual user would operate. Although this approach is common, and may be preferable over other existing types of testing, it does possess a number of rather serious shortcomings. For example, a primary weakness of this method is that, like the previously discussed manual testing method, it is bounded by human creativity and time constraints in defining and traversing usage scenarios.